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_ ms  study  tba  computational  complexity  of  finita  ver¬ 
sions  of  tba  simplest  and  fundamental  problems  of  dia- 
tributad  decision  making  and  we  show  that,  apart  from 
a  faw  axcaptions,  such  problems  are  hard  (HP-complete , 
or  worse) .  Some  of  tba  problems  studied  are  the  wall- 
known  tears  decision  problem,  the  distributed  hypothesis 
testing  problem,  as  well  as  the  problsas  of  designing 
a  communications  protocol  that  guarantees  the  attain¬ 
ment  of  a  preapecified  goal  with  aa  little  communica¬ 
tions  as  possible.  These  results  indicate  the  inherent 
difficulty  of  dieeributad  decision  making,  even  for 
very  simple  problems,  with  trivial  centralized  counter - 
carts  and  suggest  that  optimality  may  be  an  elusive 
goal  of  distributed  systems. ^ 

1.  Introduction  and  Motivation 

In  this  paper  wo  formulate  and  study  certain  simple 
decentralized  problems.  Our  goal  la  to  formulate  pro¬ 
blems  which  reflect  the  inherent  difficulties  of  dean- 

_  trelization,  that  la,  any  difficulty  in  this  class  of 

J7  problems  is  distinct  from  the  difficulty  of  correspond¬ 
ing  centralized  problems.  This  il  accomplished  by 
formulating  decentralized  problems  whose  centralized 
counterparts  are  either  trivial  or  vacioua. 

One  of  our  goals  la  to  determine  a  boundary  between 
•easy"  and  ‘hard*  decentralized  problems.  Our  results 
will  indicate  that  the  sot  of  ’easy*  problems  is 
relatively  sisal  I . 

All  problems  to  be  studied  era  Imbedded  in  a  dis¬ 
crete  framework i  the  criteria  we  use  for  deciding 
whether  e  problem  is  difficult  or  not  came  from  com¬ 
plexity  theory  [Gerey  end  Johnson,  1979 I  Papedimitrlou 
and  Steiglltz,  1982] :  following  the  tradition  of  com¬ 
plexity  theory,  problems  thst  may  Use  solved  by  a  poly¬ 
nomial  algorithm  are  considered  eeVyi  HP-cosiplete,  or 
^  worse,  proolems  are  considered  herd.*  However,  an 
w5-  ^-completeness  result  does  not  close  a  subject,  but  is. 
C —  rather  as  a  result  which  can  guide  research s  further 
t '  J>  research  should  focus  on  special  cases  of  the  problem 
^  '  or  on  approximate  versions  of  the  original  problem. 

The  main  issue  of  interest  in  decentralized  system 
.  ,  may  be  loosely  phrased  as  "who  should  communicate  to 

,  when,  what,  how  often  etc."  From  a  purely  logical 
L'i  point  of  view,  the  first  question  that  has  to  be  raised 
[  I  is  "are  there  any  communication  necessary?"  Any 
further  questions  deserve  to  be  studied  only  if  we 
y coma  to  the  conclusion  that  communications  are  indeed 
-^-.'.necessary. 

p—""  The  subject  of  Section  2  is  to  characterize  the  in- 
4 1—  he rent  difficulty  of  the  problem  of  deciding  whether 
any  cosmunicstions  are  necessary,  for  a  given  situa¬ 
tion.  we  adopt  the  following  approacht  a  decentral¬ 
ized  system  exists  in  order  to  accomplish  a  certain 
goal  which  is  extamally  specified  and  well-known.  A 


set  of  protesters  obtain  (possibly  conflicting)  obser¬ 
vations  oo  the  state  of  the  environment.  Each  processor 
has  to  make  e  decision,  based  oo  his  own  observation. 
However,  for  each  state  of  the  environment,  only  certain 
deeleiona  accomplish  the  desired  goal.  The  question 
"era  there  any  ceemuni cations  necessary?"  nay  be  than 
reformulated  as  "ean  the  goal  be  accomplished,  with  - 
certainty,  without  any  communications?"  we  show  that 
this  problem  it,  in  general,  a  hard  one. 

He  than  ia^oea  scam  more  structure  on  the  problem, 
by  assuming  that  the  observations  of  different  proces¬ 
sor!  are  related  in  e  particular  way.  The  main  issue 
that  we  address  le  "how  such  structure  is  required  so 
that  the  problem  is  an  easy  cm)'  and  we  try  to  deter¬ 
mine  the  boundary  between  easy  and  bard  problems. 

In  Section  3  we  formulate  e  few  problems  which  ere 
related  to  the  basic  problem  of  Section  2  and  discuss 
their  complexity. 

la  Section  4  we  study  a  particular  (more  structured) 
decentralised  problem  -  the  problem  of  decentre  Used 
hypothesis  casting  -  on  which  there  has  bean  sene  in¬ 
terest  recently,  and  characterise  its  difficulty. 

Suppose  thst  it  has  been  found  that  coomunicaticoa 
are  necessary.  Tha  next  question  of  interest  la  "whet 
is  the  least  amount  of  co— in  1  cations  needed?"  This 
problem  (Section  5)  is  essentially  the  problem  of  desig¬ 
ning  an  optimal  communications  protocol,  it  is  again  e 
herd  one  and  we  discuss  same  related  Issues. 

In  Section  6  we  present  our  conclusions  and  discuss 
the  conceptual  significance  of  our  results.  These  con¬ 
clusions  may  be  unmerited  by  saying  thati 

a)  Even  the  simplest  (exact)  problems,  of  decentralized 
decision  making  are  hard. 

b)  Allowing  soma  redundancy  in  communications,  nay 
greatly  facilitate  the  (off-line)  problem  of  desig¬ 
ning  e  decentralized  system. 

c)  Practical  consmnlcaticns  protocols  should  not  be  ex¬ 
pected  to  be  optimal,  as  far  as  minimization  of  the 
amount  of  caemunications  is  concerned. 

Some  of  the  results  of  this  paper  appear  in 
[Papedimitrlou  and  Tsltslklls,  1983]  and  (almost)  all 
proof*  may  be  found  in  [Tsitsiklie,  1983] . 

2.  A  Problem  of  Silent  Coordination 

In  this  section  we  formulate  end  study  the  problem 
whether  a  set  of  processors  with  diffsrsnt  information 
say  accomplish  a  given  goal  -with  certainty-  without 
any  communications. 

bat  {l . M>  be  a  set  of  processors.  Each  processor, 

eay  processor  1,  obtains  an  observation  y  which  castes 
from  e  finite  set  of  possible  observe  cions.  Then, 

processor  i  stakes  e  decision  u^  which  belongs  to  a 
finite  set  0.  of  possible  decisions,  according  to  a 


rted  by  OMR  under  contract  ONR/N00014-77-C-0532  (NR-041-S19) . 

One  way  of  viewing  HP-complete  problems ,  la  to  say  thst  thay  are  effectively  equivalent  to  the  Traveling 
Salesman  problem,  which  le  well-known  to  be  algorithmically  herd. 
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(2.1) 


following : 
Theorem  2.1; 


where  di  1*  km  function  froa  Y .  into  U. .  The 
M-tupla  (y.,...,y )  is  ths  total~inforaaEion  avail¬ 
able;  so  it  may  be  viewed  ss  ths  "stats  of  ths  envi¬ 
ronment."  For  sach  stats  of  ths  environment,  ws  assume 
that  only  csrtain  M-cuplss  (u. , . . .  .uj  of  decision  m- 
coaplish  a  given,  externally  specifies,  goal.  Hors 
precisely,  for  sach  (v^ , . . . ,  y^)  €  Y^x-.-xY^we  ars  givan 

a  sat  3(y,»...,y  M)  C  OjX...xOM  of  satisficing  decisions. 

(So,  S  aay  ba  viswsd  as  a  function  froa 
0.x. .  .x0 

Y, xY  x...x  Y  into  2  *  H. 

12a 

Ths  problaa  to  bs  studied,  which  ws  call  "distri¬ 
buted  satisficing  probin'  (after  ths  tara  introduced 
by  H.  Simon  [1980] )  aay  bs  described  formally  as 
follows: 

Distributed  Satisficing  (OS) :  Given  finite  sets  Y^, .... 
YH,  U^, .. .  .Ojj  and  a  function  S:  Y^  x...x  Y^  ♦ 

O  x. ..x  uM 

2  ,  are  there  functions  3^:  Y^  i"l, 

2, . . .  ,M,  such  that 

l3l(ylJ * •••* 3MlyH5)estyi‘  •  *  Vty^. ...»y  >« 

Yt  X...X  Ym  (2.2) 

Remarks: 

1.  Me  are  assuming  that  the  function  S  is  "easily 
computable ; *  for  example,  it  may  be  given  in  the  fora 
of  a  table. 

2.  The  centralized  counterpart  of  DS  would  ba  to 

allow  the  decision  u  of  each  agent  depend  on  the 
entire  set  (y^ . y^)  of  observations:  so,  would 

be  a  function  froa  Y1  x...x  Y^  into  0^.  (This  cor¬ 
responds  to  a  situation  in  which  all  processors  share 
the  same  information.).  Clearly,  then,  there  exist 
satisfactory  (satisficing)  functions  ^ iY^x. ..xY^ -"0^, 

if  and  only  if  Sly^...^)^,  Y<YX . yM,eYllt**‘*YM* 

Since  S  is  an  "easily  computable"  set  as  a  function 
of  its  arguments,  we  can  see  that  ths  centralized 
counterpart  of  DS  is  a  trivial  problem.  So,  any  dif¬ 
ficulty  inherent  in  DS  is  only  caused  by  the  fact  that 
information  is  decentralized. 

3.  A  "solution”  for  the  problem  DS  cannot  be  a  closed- 
form  formula  which  gives  an  answer  0(no)  or  l(yes). 
Rather,  it  has  to  be  an  algorithm,  a  sequence  of  ins¬ 
tructions,  which  starts  with  the  data  of  the  problaa 
IY^, ...,Y  ,  0^, ... ,U^,S)  and  eventually  provides  the 

correct  answer.  Accordingly,  the  difficulty  of  the 
problem  DS  may  ba  characterized  by  determining  the 
place  held  by  DS  in  the  complexity  hierarchy.  For 
definitions  related  to  computational  complexity  and 
the  methods  typically  used,  the  reader  is  referred  to 
[Carey  and  Johnson,  1979:  Papadimitriou  and  Steiglitz, 
1982] . 

4.  If,  for  some  i,  the  set  IT  is  a  singleton,  proces¬ 
sor  i  has  no  choice,  regarding  his  decision  and,  con¬ 
sequently,  the  problem  is  equivalent  to  a  problem  in 
which  processor  i  is  absent.  Hence,  without  loss  of 
generality,  we  only  need  to  study  instance,  of  DS  in 
which  |oil>  2, 

5.  He  believe  that  the  problem  DS  captures  the  es¬ 
sence  of  coordinated  decision  ashing  with  decentral¬ 
ized  information  and  without  coonwications  (silent 
coordination) . 

Some  initial  results  on  DS  are  given  by  the 


a)  The  problem  DS  with  two  processors  (M>2)  and  res¬ 
tricted  to  instances  for  which  the  cardinality  of  the 
decision  sets  is  2  (|uj*2,  1*1,2)  aay  be  solved  in 

polynomial  time. 

b)  The  problem  DS  with  two  processors  (M>2)  is  HP- 
complete,  even  if  we  restrict  to  instances  for  which 
IUJ-2.  |02|-3. 

e)  The  problem  DS  with  three  (or  more)  processors 
(M>3)  is  NF-cooplete,  even  if  we  restrict  to  instances 
for  which  |u^|»2,vi 

Theorem  2.1  states  that  the  problem  DS  is,  in  gen¬ 
eral,  a  hard  combinatorial  problem,  except  for  the 
special  case  in  which  there  are  only  two  processors 
and  each  one  has  to  make  a  binary  decision.  It  should 
be  noted  that  the  difficulty  is  not  caused  by  an  at¬ 
tempt  to  optimize  with  respect  to  a  cost  function, 
because  no  cost  function  has  been  introduced.  In  game 
theoretic  language,  we  are  faced  with  a  "game  of  kind," 
rather  than  a  "game  of  degree." 

We  will  now  consider  some  special  cases  (which  re¬ 
flect  the  structure  of  typical  practical  problems)  and 
examine  their  computational  complexity,  trying  to  deter 
mine  the  dividing  line  between  easy  and  hard  problems. 
From  now  on  we  restrict  our  attention  to  the  case  in 
which  there  are  only  two  processors.  Clearly,  if  a 
problem  with  two  processors  is  hard,  the  corresponding 
problem  with  three  or  more  processors  cannot  be  easier. 

He  have  formulated  above  the  problem  DS  so  that  all 
pairs  (y^.y^ja  Y^xY^  are  likely  to  occur,  so,  the 

information  of  different  processors  is  completely  un¬ 
related:  their  coupling  is  caused  only  by  the  structure 
of  the  satisficing  sets  S(y^,y2).  In  most  practical 

situations,  however,  information  is  not  completely  uns¬ 
tructured:  when  processor  1  observes  y^,  he  is  often 

able  to  make  certain  inferences  about  the  value  of  the 
observation  y.  of  the  other  processor  and  exclude  cer¬ 
tain  values.  He  now  formalize  these  ideas: 

Definition:  An  Information  Structure  I  is  e  subset 
of  YjXY^.  He  say  that  an  information  structure  I  has 

degree  (D, ,D2)  (D^.D^  are  positive  integers)  if 

(i)  For  each  y^SY^  there  exist  at  most  D^  distinct 
elements  of  Y2  such  that  (y^.yjJe  I. 

(ii)  For  sach  Y2e*2  th,ra  •xi*t  at  most  D,  distinct 
elements  of  Y^  such  that  (y^.Yjie  I. 

(iii)  D^.Dj  ***  smallest  integers  satisfying  (i) , 

(ii) .  An  information  structure  I  is  called  classical 
if  DjWDjWlr  nested  if  D^*l  or  Dj»l. 

We  now  interpret  this  definition:  The  information 
structure  I  is  the  set  of  pairs  (y^'Yj)  ot  observations 

that  aay  occur  together.  If  I  has  degree  (O^.D^) 

processor  1  may  use  his  own  observation  to  decide  which 
elements  of  Y2  aay  have  been  observed  by  processor  2. 

In  particular,  ha  aay  exclude  all  elements  except  for 
D.  of  them.  The  situation  faced  by  processor  2  is 
synoetrical. 

If  D^"l  and  processor  1  observes  y^,  there  is  only 

one  possible  value  for  y  .  So,  processor  1  knows  the 
observation  of  processor2!.  (The  converse  is  true 
when  D  >1) .  This  is  called  a  nested  information  struc¬ 
ture  because  the  information  of  one  processor  contains 
th»  information  of  the  other. 

When  D  “Oj"!,  each  processor  knows  the  observation 
of  the  other:  so,  their  information  is  essentially 


3.  Related  Problems 


ihind. 

Sine*  p*ir*  (y  ,y  )  not  in  I  cannot  occur,  eh*r*  is  no 
meaning  in  requiring 2 th*  processors  to  ask*  compatible 
decisions  if  (y^.y^)  were  to  be  observed.  This  leads 

to  the  following  version  of  the  problem  DS: 

OS  I :  Given  finite  sets  Y, ,Y,,U. ,0,,  1C  Y,xY,  and  a 
-  0X0  12*2  *  2 

function  S:  1*2  1  ,  are  there  functions 

1*1,2,  such  that 

(d1Cy1).ajty2))«s(y1,y23,  Yty^y^dx?  .  (2.3) 

Note  that  any  instance  of  DSX  is  equivalent  to  an  ins¬ 
tance  of  OS  in  which  Sty^.yj)-  U^xUj,  Vty^.yjJS  1. 

That  is,  no  compatibility  restrictions  are  placed  on 
the  decisions  of  the  two  processors,  for  those  (y,>y.) 
that  cannot  occur. 

We  now  proceed  to  the  main  result  of  this  Section: 
Theorem  3.3.3: 

a)  The  problem  DSX  restricted  to  instances  satisfying 
any  of  the  following: 

(i)  On*  or  more  of  |0jJ,|u2|,  D^.Dj  is  equal  to  1. 
tii)  |o1|-|o2|-2. 

(iii)  Dj-Dj-2, 

(iv)  Dj-IUjl-2,  (or  Dj-luJ-2) 
say  be  solved  in  polynomial  time. 

b)  The  problem  DSX  is  NF-campleta  even  if  we  restrict 
to  instances  for  which 

loj-  Oj-3.  |02|-02-2 

The  result  concerning  the  case  D^-l  or  02"1  is  not 

surprising.  It  is  well-known  that  nested  information 
structures  may  be  exploited  to  solve  otherwise  dif¬ 
ficult  decentralized  problems.  But  except  for  the  case 
D^Dj-2  (which  is  sort  of  a  boundary)  the  absence  of 

nestedness  makes  decentralized  problems  computationally 
hard.  Our  result  gives  a  precise  meaning  to  the  state¬ 
ment  that  non-nested  information  structures  are  much 
more  difficult  to  handle  than  nested  ones. 

Theorem  3.2.2  shows  that  even  if  D^.Dj  are  held  cons¬ 
tant,  the  problem  DSX  is.  in  general,  NP-complsts. 

There  is,  however,  a  special  ease  of  DSX,  with  D^,D2 

constant,  for  which  an  efficient  algorithm  of  the 
dynamic  programming  type  is  possible: 

Theorem  3.2.3:  Let  Yj"{l,.. . .  ,m},  Y2»{l,...,n}  and  sup¬ 
pose  that  |i-j|<  0.  Y(i,j)«  X.  Then,  if  D  is  held 
constant,  DSX  may  be  solved  in  polynomial  time. 

Remark:  In  fact,  the  conclusion  of  Theorem  3.2.3  re¬ 
mains  true  if  w*  assume  mm  and  we  replace  the  condition 
|i-j|<_  D  by  the  weaker  condition  |i-j|  (mod  n)<  D. 

The  proof  consists  of  a  small  modification  of  the 
preceding  on*. 

The  condition  | i-3  i  <_  D,  Y(i,j)6  I  is  fairly  natural 
in  certain  applications.  For  example,  suppose  that  the 
observations  y^  and  y2  are  noisy  measurements  of  an 

unknown  variable  x  (y^x+v^  where  the  noises  w^  are 

bounded:  |w^|^D/2. 

The  condition  |i-j| (mod  n)<_  D  may  also  arise  if  the 
observations  y^.yj  **•  noisy  measurements  of  some 

unknown  angle:  y^m9  *  w^ 


In  this  Section  we  define  and  discuss  briefly  a  few 
more  combinatorial  problems  relevant  to  decentralized 
decision  making.  All  of  them  will  be  seen  to  be  harder 
than  problem  DS  of  the  last  section  (i.e.  they  contain 
IS  as  a  special  case)  and  are,  therefor*  NP-hard  (that 
is,  N7-complet*,  or  worse). 

The  best  known  static  decentralized  problem  is  the 
team  decision  problem  [Marschak  and  Badner,  1972] 
which  admits  an  elegant  solution  under  linear  quadratic 
assumptions.  Its  discrete  version  is  the  following: 

TOP  (Team  Decision  Problem) :  Given  finite  sets  Y^, Y2 
Ul'U2'  *  probability  mass  function  p:  Y^xYj-Q,  and  a 
cost  function  c:  Y^xYjXO^xOj  »M,  find  decision  rules 
Y  *  0  ,  i“l,2  which  minimize  the  expected  cost 

2  2  c<y1*y2*S1(y1)  »32(y2))p(y1*y2) 

yl*Yl  y2€Y2 

Let  S(y1,y2>-{(u1,u2)€U1xB2:  c^.yj.u^u^-O}.  Xf 

we  solve  TDP,  w*  have  effectively  answered  the  question 
whether  there  exist  d2>d2  such  that  J(d2>d2)a°>  This 

is  equivalent  to  the  question  whether  there  exist  sat¬ 
isficing  decision  rules  (with  the  satisficing  sets 
S (yi,y2)  defined  as  above).  Therefore,  TDP  is  harder 

than  DS: 

Proposition  3.1:  The  discrete  team  decision  problem 

is  KP-hard,  even  if  the  range  of  the  coat  function 
c  is  {0,1}. 

Instead  of  trying  to  'satisfice*  for  every  pair  of 
observations  ly^>y^)8  Y^.xYj,  it  may  be  more  appro¬ 
priate  to  impose  a  probability  mass  function  an  Y^xY^ 

and  try  to  maximize  the  probability  of  satisficing. 

This  leads  to  th*  next  problem: 

KPS  (Maximize  Probability  of  Satisficing) :  Given 
finite  sets  *j,'X2'D1'D2'  *  probability  mass  function 

0  xU 

p:  YjXYjVQ  and  a  function  S:  Y^xYj  ♦  2  1  2,  find 

decision  rules  Y^—u^,  1*1,2,  which  maximize  the 
probability  of  satisficing  J(d^,d2)  » 

PrUb^yj)  >a2 (yJ))es(yi,y2) ) . 

We  now  take  a  slightly  different  point  of  view. 
Suppose  that  communications  are  allowed,  so  that  the 
processors  may  always  make  satisficing  decisions  by 
comunieating  (assuming  that  S (y^.Yj)^, 

V<yi<y2>e  y^xy^) .  Suppose,  however,  that  communica¬ 
tions  are  very  expensive,  so  that  w*  are  interested 
in  a  scheme  which  guarantees  satisficing  with  a  mini¬ 
mum  amount  of  communications,  w*  will  assume  that  if 
on*  of  th*  processors  initiates  a  communication,  all 
their  information  will  be  exchange  at  unity  cost. 

(For  a  more  refined  way  of  counting  th*  amount  of  com¬ 
munications,  see  Section  3.S.) 

MPC  (Minimize  Probability  of  Communications) :  Given 
finite  sets  Y^, *2'°i'°2  *  probability  mass  function 

0  XU 

p:  Y^xYj  *  Q  and  a  function  S:  Y^xYj»2  ,  find 
decision  rules  Y^-U^lKc},  i*l,2,  which  minimize 
the  probability  PrJd^ty^-C  or  d2<y2)aC)  of  communica¬ 
ting  subject  to  th*  constraint 


If  and  S^y^X]  then  (d1(y1),dJ(y2))esiy1,y2>. 

The  proof  of  the  following  is  trivial: 

Proposition  3.2;  The  pros Isms  MPS  and  MPC  art  NP-hard. 
In  fact,  ws  also  hava: 

Proposition  3.3:  The  problems  TDP  (with  a  zaro-ona 
cost  function)  and  MPS  a ra  NP-hard,  aven  if  1 0^ j “ 1 J*2  . 

Wa  could  also  dafina  dynamic  varsions  of  OS  or  of 
tha  taaa  problem,  in  a  straightforward  way  (Tannay , 

1983] .  Sinca  dynamic  problams  cannot  ba  aasiar  than 
static  onas,  thay  ara  automatically  NP-hard. 

4.  Oacantralizad  Hypothasis  Tasting 

A  basic  problam  in  dacantralizad  signal  procassing, 
which  has  attractad  a  fair  amount  of  attantion  racantly, 
is  tha  problam  of  dacantralizad  hypothasis  tasting 
[Tannay  and  Sandall,  1981;  Ekchian,  1982;  EX chi an  and 
Tannay,  1982;  Kushnar  and  Pacut,  1982;  Lauar  and 
Sandall,  1983].  A  siapla  varsion  of  tha  problam,  in¬ 
volving  only  two  processors  and  two  hypotheses  may  ba 
described  as  follows: 

Two  processors  S^  and  S2  receive  observations  y^eY^, 
y2«Y2,  respectively,  where  Y^  is  tha  sat  of  all  pos¬ 
sible  observations  of  processor  i.  (Figure  1) .  Thera 
ara  two  hypotheses  and  on  tha  state  of  the 

anvironmant ,  with  prior  probabilities  p  and  p,  res- 

O  1 

pectivaly.  For  each  hypothesis  wa  ara  also  givan 
tha  joint  probability  distribution  P(y1,y2|si)  of  tha 

observations,  conditioned  on  the  event  that  R.  is  true. 
Upon  receipt  of  y^,  processor  evaluates  a  massage 

u^sto.l}  according  to  tha  rule  (y^) ,  where 

S^Y^O.l}.  Than,  and  u2  are  transmitted  to  a 


central  processor  (fusion  canter)  which  evaluates 
u  -u,n  u  and  daclaras  hypothasis  H  to  ba  true  if 

u  -0,  H.  if  u  »1.  (So,  wo  ••••ntially  havo  a  voting 
0X0 

scheme).  Tha  problam  is  to  select  tha  functions  3^< 

3,  so  as  to  minimize  the  probability  of  accepting  the 
wrong  hypothasis.  (Mora  general  par fozmanca  criteria 
may  ba  also  considered) . 

Most  available  results  assume  that 

P<yl'y2lHi>“P(yliHi)P(y2*Hi) '  i"1'2'  <4-1) 

which  states  that  the  observations  of  tha  two  proces¬ 
sors  are  independent,  when  conditioned  on  either  hy¬ 
pothesis.*  In  particular,  it  has  been  shown  (Tenney 
and  Sandell,  1981]  that  the  optimal  decision  rules  3^ 

are  given  in  terms  of  thresholds  for  the  likelihood 

P0P(Holyi) 

ratios  -  — ■—  .  The  optimal  thresholds  for 

(h^  j  y^; 

the  two  sensors  are  coupled  through  a  system  of  equa¬ 
tions  which  gives  necessary  conditions  of  optimality. 
(These  equations  are  precisely  tha  person-by-person 
optimality  conditions) .  Few  analytical  results  are 
available  when  the  conditional  independence  assumption 
is  removed  [Lauar  and  Sandell,  1983].  The  approach  of 
this  section  is  aimed  at  explaining  this  status  of  af¬ 
fairs,  by  focusing  on  discrete  (and  finite)  versions 
of  tha  problem. 

We  first  have: 

Theorem  4.1:  If  Y^.Yj  are  finite  sets  and  (4.1)  holds, 

then  optimal  choices  for  3, >3,  may  be  found  in  poly¬ 
nomial  time. 

So,  under  tha  conditional  independence  assumption, 
decentralized  hypothesis  testing  is  a  computationally 
easy  problam.  Unfortunately,  this  is  not  the  case  then 
tha  independence  assumption  is  relaxed.  Our  main  re¬ 
sult  (Theorem  4.2)  states  that  (with  Y^,  Y2  finite 

sets) ,  decentralized  hypothesis  tasting  is  a  hard  com¬ 
binatorial  problam  (NP-hard) .  This  is  true  even  if  we 


uosu1nu2 


Figure  1:  A  Scheme  for  Decentralized  Hypothesis 
Testing. 


restrict  to  the  special  case  where  perfect  detection 
(zero  probability  of  error)  is  possible  for  the  corres¬ 
ponding  centralized  hypothesis  testing  problem. 

Although  this  is  in  same  sense  a  negative  result,  it  is 
useful  because  it  indicates  tha  direction  in  which 
future  research  on  this  subject  should  proceed: 

Instead  of  trying  to  find  efficient  exact  algorithms, 
research  should  focus  on  approximate  algorithms,  or 
exact  algorithms  for  problems  with  more  structure  than 
that  assumed  here.  Moreover,  our  result  implies  that 
any  necessary  conditions  for  optimality  to  be  developed 
ara  likely  to  be  deficient  in  one  of  two  respects: 

a)  Either  there  will  be  a  very  large  number  of  deci¬ 
sion  rules  satisfying  these  conditions. 

b)  Or,  it  will  be  hard  to  find  decision  rules  satis¬ 
fying  these  conditions. 

In  particular,  optimal  decision  rules  are  not  given  in 
terms  of  thresholds  on  likelihood  ratios. 

Of  course,  there  remains  the  question  whether  ef¬ 
ficient  approximate  algorithms  exist  for  the  general 
decentralized  hypothesis  testing  problem,  or  whether 
we  must  again  restrict  to  special  cases  of  the  problem. 
We  now  present  formally  the  problem  to  be  analyzed. 

DKT:  (Decentralized  Hypothesis  Testing,  Restricted  to 
Instances  for  which  Perfect  Centralized  Detection  is 
Possible) . 

We  are  given  finite  sets  Y^,Y2;  a  rational  number 

number  k;  a  rational  probability  mass  function 
p:  Y^xYj  **QO  [0,1] ;  a  partition 

*  Such  an  assumption  is  reasonable  in  problems  of  detec¬ 
tion  of  a  known  signal  in  independent  noise,  but  is  typ- 
iignai  in  problams  of  detection  of  an  unknown 


{a  i  A, }  of  Y,x Y,.*  Do  there  exist  3,  :Y,»{0,1}, 

0  1*2  11 

such  that  where 

-  (d,  -d,)  -  £  *<y,.Y,)d,  (yja.ly,)  ♦ 

A  2  (Jvy2)eAo  * 

(y,?y2)«A1  '^hi-WW17  (4.2) 

Remarks:  1.  If  we  let  k-Q ,  than  DHT  is  a  special  csss 
of  problem  OS  (Section  2),  with  1 0^ j “ | j «2 ,  and  is 

polynomially  solvable,  according  to  Thaoraa  3. '2.1.  In 
general  DHT  is  a  special  ease  of  UPS  and  TOP  (Section 
3.3)  with  lu1l*lu2l“2'  Consequently,  Theorem  4.2 

below  proves  Proposition  3.3. 

2)  Clearly,  the  optimization  problem  (Minimize 

with  respect  to  cannot  be  easier  than  DHT. 

Since  DHT  will  be  shown  to  be  N7 -complete ,  it  follows 
that  the  above  optimization  problem  is  NP-hard. 

3)  In  DHT,  as  defined  above,  we  are  only  considering 
instances  for  which  perfect  centralized  detection  is 

possible:  Think  of  H  as  being  the  hypothesis  that 
o 

(y,  ,y je  A  ,  and  H,  as  being  the  hypothesis  that 
12  0  1 

(y,  <y2)S  A^.  Certainly,  if  a  processor  knows  both  y^, 

y2>  the  true  hypothesis  may  be  found  with  certainty. 

For  the  decentralized  problem,  the  cost  function 
2^,3  )  is  easily  seen  to  be  the  probability  of  error. 

4)  The  result  to  be  obtained  below  remains  valid  if  dm 

fusion  center  uses  different  rules  for  combining  the 
messages  it  receives  (e.g.  u0” 1  \v <  ) ) »  or  if  we 

leave  the  combining  rule  unspecified  and  try  to  find 
an  optimal  combining  rule. 

Theorem  4.2:  DHT  is  HP-complete. 

5.  On  Designing  Communications  Protocols 

Suppose  that  we  are  given  an  instance  of  the  dis¬ 
tributed  satisficing  problem  (DS)  and  that  it  was 
concluded  that  unless  the  processors  coenunicate, 
satisficing  cannot  be  guaranteed  for  all  possible  ob¬ 
servations.  Assuming  that  cosnunications  are  allowed 
(but  are  costly) ,  we  have  to  consider  the  problem  of 
designing  a  communications  protocol:  what  should  each 
processor  communicate  to  the  other,  and  at  what  order? 
Moreover,  since  communications  are  costly,  we  are 
interested  in  a  protocol  which  minimizes  the  total 
number  of  binary  messages  (bits)  that  have  to  be  com¬ 
municated.  (The  word  "bits"  above  does  not  have  the 
information  theoretic  meaning.) 

Before  proceeding,  we  must  make  more  precise  the 
notion  of  a  conmmnication  protocol  and  of  the  number 
of  bits  than  guarantee  satisficing. 

Given  an  instance  of  the  problms 

DSI  we  will  say  that: 

Thera  is  a  protocol  which  guarantees  satisficing 
with  0  bits  of  communications,  if  V  is  a  YES  instance 
of  the  problem  DSI.  (That  is,  if  there  exist  satis¬ 
ficing  decision  rules,  involving  no  communications . ) 
we  then  proceed  inductively: 

There  is  e  protocol  which  guarantees  satisficing 
with  K  bits  of  communications  (xe  N) ,  if  for  some 
ifi{l,2}  (say,  i«l)  there  is  a  function  m:Y  -Ho,l}, 
such  that  for  each  of  the  instances.  * 

0<-(yinm*i(0),r2.01,02,I  n((Yinm"i(0))xY2J,S)  and 

*  That  is  A^Aj-YjXYj  and  A^A^.  “  ~~~  “— 


■(Y^i 


(1)  ,Y2,01,U2,I  OtYjOrn  (1)  )xYj]  ,S)  there  is 


a  protocol  which  guarantees  satisficing  with  not  more 
than  K-l  bits  of  communications.  (Here  m”l(i)« 
{yieY1:m(y1)-i>.) 

The  envisaged  sequence  of  events  behind  this  defini¬ 
tion  is  the  following:  Each  processor  observes  his 
measurement  y^6Y^,  i-1,2.  Then,  one  of  the  processors, 

say  processor  1,  transmits  a  message  m(y, ),  with  a 
single  bit  to  the  other  processor,  From*that  point  on, 
it  has  become  consnon  knowledge  that  y^TY^n  m~2  (y  )  • 


therefore,  the  remaining  elements  of  Y.  may  be  ignored, 
we  can  now  state  formally  the  problem  of  interest: 


MBS  (Minimum  bits  to  satisfice):  Given  an  instance  V 


of  DSI  and  K£  N,  is  there  a  protocol  which  gurantees 
satisficing  with  not  more  than  X  bits  of  communications? 

By  definition,  MBS  with  K-0  is  identical  to  the 
problem  DSI.  Moreover,  MBS  with  X  arbitrary  cannot  be 
easier  than  MBS  with  K-0  (which  is  a  special  case) . 
Therefore,  MBS  is,  in  general  HP-hard.  Differently 
said,  problems  involving  caeznuni cations  are  at  least 
as  hard  as  problems  involving  no  communications . 

We  have  seen  in  Section  2  that  when  |u2|w|o2|<>2, 

DSI  may  be  solved  in  polynomial  time .  Therefore,  MBS 
with  K*0,  |  UjJ "2 ,  1 1?2 ]  “2  is  polynomially  solvable. 

However,  for  arbitrary  X,  this  is  no  longer  true: 


Theorem  S.li  MBS  is  HP-complete,  even  if  | 0^|“| 02|“ 

{0,1}  and  even  if  we  restrict  to  instances  for  which, 
for  any  (y^y^ei,  either  S(y1,y2)»  {(0,0)}  or 


S(y1»y2)-{(1.1)}. 


The  above  theorem  proves  a  conjecture  of  A.  Yao 
[Yao,  1979) .  The  proof  was  mainly  constructed  by 
C.  Papadimitriou  and  may  be  found  in  (Papadimitriou 
and  Tsitsiklis,  1982) . 

We  should  point  out  that  the  special  caae  referred 
to  in  Theorem  5.1  concerns  the  problem  of  distributed 
function  evaluation:  we  are  given  a  Boolean  function 
f  :Y.xY2 {0,1}  and  we  require  that  both  agents  (proces¬ 
sors  eventually  determine  the  value  of  the  function 


(given  the  observation  -input  (y^.y^),  by  exchanging 


a  minimum  number  of  bits.  In  our  formalism: 
Slyl'y21”  ^(0’0)  }  e<Y1'Y2}“0  “d  s  (Y]/Y2)”{  (1,1) } 


if  fCy^YjJ-l. 


In  Section  2  we  had  investigated  the  complexity  of 
DSI  by  restricting  to  instances  for  which  the  set  I 

This  may  be  done,  in 


had  constant  degree  (D^.Dj) 


principle,  for  MBS,  as  well,  but  no  results  are  avail 


able,  except  for  the  simple  case  in  which  D^»D2-2. 


In  fact,  when  D^*D  “2  each  processor  may  transmit 
his  information  to A the  other  agent  by  connunicating  a 
single  binary  message  and,  for  this  reason,  we  have: 


Proposition  5.1:  KBS  restricted  to  instances  for  which 
D  »D  «2  may  be  solved  in  polynomial  time.  Moreover, 
an  optimal  protocol  requires  transmission  of  at  most 
two  binary  messages,  one  from  each  processor. 


When  (D^fDj)  is  larger  than  (2,2),  there  is  not 


much  we  can  say  about  optimal  protocols.  However,  it 
is  easy  to  verify  that  there  exist  fairly  simple  non¬ 
op  tiaal  protocols  (which  may  be  calculated  in  poly¬ 
nomial  time)  which  involve  relatively  small  amounts  of 
eoootunication.  This  is  because: 


Proposition  5.2:  Suppose  that  I  has  degree  (D^.Dj)  and 

that  S(y  ,y  W.  Vly^.yjjei.  Then  intonation  may  be 

centralized  (and  therefore  satisficing  is  guaranteed) 
by  means  of  a  proeocol  requiring  eoomunicetion  of  at 


j  binary  messages  by  aach  processor. 

Moreover.  such  a  protocol  aay  ba  constructed  in  tisa 
0(C|Y1[*iY2|H|Y1l+|Y2|)).  (Kara  |xj  ,  x€  R,  stands 

for  the  smallest  intagar  larger  than  x.) 


Remark  1:  It  sight  ba  tempting  to  guaaa  that  procts- 

sor  I  (respectively  2)  naada  to  conunicate  only 
[log202J  (raapactivaly  jlog^J)  bit*,  but  thia  ia  r.ot 

trua,  aa  can  ba  aaan  from  fairly  simple  examples. 

6.  Conclusion* 

Ha  sunnariz*  hara  tha  sain  concluaiona  of  thia  pa- 
par. 

Evan  if  a  sat  of  procaaaora  have  complete  knowladga 
of  tha  structure  of  a  diatributad  daclaion  making  pro- 
blaa  and  tha  daairad  goal;  avan  if  tha  corrasponding 
centralized  problaa  ia  trivial;  avan  if  all  ralavant 
sacs  are  finite,  a  satisficing  decision  rule  that  in¬ 
volves  no  on-line  cosasun  1  cations  aay  be  vary  hard  to 
find,  the  corresponding  problaa  being,  in  general, 

HP -comp let*.  There  are  many  objections  to  tha  idea 
tha  HP -completeness  is  an  unequivocal  measure  of  tha 
difficulty  of  a  problem,  because  it  ia  based  on  a 
worst  ease  analysis,  whereas  the  average  performance 
of  an  algorithm  might  be  a  more  adequate  measure; 
moreover  NP-hard  optimization  problems  say  have  very 
simple  approximate  algorithms.  However ,  HP-complete 
problems  are  often  characterized  by  the  property  that 
any  known  algorithm  is  "'ary  close  to  systematic  ex¬ 
haustive  search;  they  do  not  poaaeas  any  structure  to 
be  exploited. 

Concerning  the  problem  DS,  and  its  variations,  we 
may  reach  the  following  specific  conclusions:  No  sim¬ 
ple  algorithm  could  solve  DS.  Given  that  coaiwilca- 
tiona  would  ba  certainly  required  for  thoie  instances 
of  DS  that  possess  no  satisficing  decision  rules,  it 
would  not  be  s  great  loss  if  we  allowed  tha  proces¬ 
sors  to  conminicate  even  for  some  instances  of  DS  for 
which  this  would  not  be  necessary.  Even  if  these 
extra  communications  -being  redundant-  do  not  lead  to 
better  decisions,  they  may  greatly  facilitate  the  de¬ 
cision  process  and  -from  a  practical  point  of  view  - 
remove  some  load  from  the  computing  machines  employed. 

Concerning  the  problem  of  distributed  hypothesis 
testing,  we  have  shown  that  it  becomes  hard,  once  a 
simplifying  assumption  of  conditional  independence  ia 
removed.  This  explains  why  no  substantial  progress 
on  this  problem  had  followed  the  work  of  Tenney  and 
Sandell  [1982]. 

From  a  more  general  perspective,  we  are  in  a  posi¬ 
tion  to  say  that  the  basic  (and  the  simplest)  problems 
of  decentralized  decision  making  are  hard,  in  a  precis# 
mathematical  sense.  Moreover,  their  difficulty  does 
not  only  arise  when  one  is  interested  in  optimality. 
Difficulties  persist  even  if  optimality  is  rsplsead 
by  satisficing.  As  a  consequence,  further  research 
should  focus  on  special  cases  and  easily  solvable 
problem  as  well  as  on  approximate  versions  of  the 
original  problems . 

In  casas  where  communications  are  necessary  (but 
costly)  there  arises  naturally  the  problem  of  desig¬ 
ning  a  protocol  of  coessunications .  Unfortunately,  if 
this  problem  is  approached  with  the  intention  to  mi¬ 
nimize  the  amount  of  communications  that  will  guar¬ 
antee  the  accomplishment  of  a  given  goal,  wa  art  again 
led  to  intractable  combinatorial  problems.  Therefore, 
praceical  communications  protocols  can  only  ba  desig¬ 
ned  on  a  'good*  heuristic  or  ad-hoc  basis,  and  they 
should  not  be  expected  to  be  optimal;  approximate 
optimality  Is  probably  a  more  meaningful  goal.  Again, 
allowing  soma  redundancy  in  on-line  coemtuni  cat  ions 
My  lead  to  substantial  savings  in  off-line  computa¬ 
tions. 
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